IMAGE-PROCESSING METHOD, IMAGE -PROCESSING APPARATUS, 
AND IMAGE- PROCESSING PROGRAM PRODUCT 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 
[0001] 

The present invention relates to an image-processing 
method, an image-processing apparatus, and an image-processing 
program product. 

2. Description of the Related Art 
[0002] 

With the recent improvement of performance of ink jet 
printers and digital cameras, it has become possible for us to 
print images in the same quality as silver-film photographs. 
The subj ects of printing here are digital images, which can easily 
be retouched; accordingly, we usually retouch images for higher 
quality by using photo retouch software, printer drivers, etc. 
As such retouch requires proficiency of a certain degree, various 
kinds of automatic retouching have been realized. For example, 
the quantities of characteristics are extracted from an image, 
the tendencies of gradation of the image are grasped by analyzing 
the quantities of characteristics, and the data of the image 
are revised to change the tendencies of gradation (See, for 
example, Japanese Unexamined Patent Publication No. 
2000-36043) . 
[0003] 

The above prior art has the following problem. If an 
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artificial image is added to a natural image, the prior art grasps 
the tendencies of gradation of the natural image wrongly and 
processes the natural image inappropriately . 

An artificial image of a letter, line> pattern, or the like 
is often composed by collecting a plurality of pixels of a single 
color. Accordingly, if the prior art grasps the tendencies of 
gradation (for example, the histogram of brightness) of a natural 
image containing such an artificial image, tendencies of 
gradation different from those of the natural image are grasped 
because of the high frequency of appearance of a certain value 
of brightness and so on. Accordingly, the prior art makes 
inappropriate processing or processing of an inappropriate 
degree (the degree of revision, etc.) to the natural image. 

SUMMARY OF THE INVENTION 
[0004] 

In accordance with the above, it is the obj ect of the present 
invention to provide an image-processing method, an 
image-processing apparatus, and an image-processing program 
product capable of processing natural images containing 
artificial images appropriately. 

To achieve the object, such a portion or portions of an 
artificial image or images contained in a natural image as 
characterize the artificial image or images are extracted from 
the natural image and the whole image excluding the portion or 
portions are processed according to the present invention. 
Accordingly, when the natural image is analyzed, the 
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characteristic portion or portions of the artificial image or 
images do not affect the analysis and, hence, the natural image 
can be processed appropriately. For example, the tendencies 
of gradation of the natural image are grasped appropriately and 
the natural image is processed appropriately in accordance with 
the tendencies of gradation. 
[0005] 

A means of extracting artificial images will serve the 
purpose if it is capable of extracting an artificial image or 
images, which are contained in a natural image, from the natural 
image; accordingly, the means can be of various types of 
construction. Because the image data of an artificial image 
reveal certain characteristics, the portion or portions 
characterizing the artificial image can be identified by 
extracting characteristic data from the image data of the 
artificial image . 
[0006] 

To extract image data characteristic of an artificial image, 
various characteristics can be used. For example, the color 
of each pixel may be used. In this case, a portion consisting 
of a plurality of successive pixels of a certain prescribed color 
is extracted and the portion is regarded as an artificial image. 
If one is to add an artificial image of letters, lines, patterns, 
etc. to a natural image, one prepares the artificial image with 
application software which can be run on a computer and overlays 
the artificial image on the natural image. 
• [0007] 
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Numerous colors (for example, about 16,700,000 colors) 
can be expressed in an image being processed by a computer, whereas 
the colors of letters, etc. to be preparedby application software 
are usually only a limited part of the numerous colors. 
Accordingly, colors to be used by many kinds of application 
software are designated as specific colors in advance. Most 
artificial letters consist of a plurality of successive pixels. 
On the other hand, it is very rare for a plurality of successive 
pixels of the same color to appear in a natural image even if 
any part may appears to human eyes to be in a single color. 
[0008] 

Accordingly, artificial letters, etc. can easily be 
extracted by finding a plurality of successive pixels of one 
of the above specific colors . The colors often used by the above 
application software are, for example, those colors which are 
formed by combining common specific values with color component 
values . For example, there are many application programs which 
allow users to choose, as the colors of letters, colors expressed 
by the combinations of RGB color component values and the specific 
values of 0, 64, 128, 192, and 255 in a computer which expresses 
about 16, 700, 000 colors by combining RGB color component values 
and gradients of 0 to 256. Therefore, the colors (R, B, G) = 

(0, 0, 0), (0, 0, 64), . . ., (255, 255, 255) expressed by the 
combinations of RGB color components and the specific values 
of 0, 64, 128, 192, and 255 can be designated as specif ic colors . 

[0009] 

It is preferable to determine whether the color of a pixel 
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is one of the specific colors or not by determining whether the 
color component values of the pixel are exactly equal to the 
color component values of the specific color, because this method 
allows the pixels of specific colors to be extracted by simple 
processing. It is preferable in some cases, however, to regard 
the color of a pixel as one of the specific colors, allowing 
a margin of error of 1 or 2. For example, expansion into the 
basic function is made by Fourier transformation in the 
compression of an image by JPEG; accordingly, two pixels with 
the same color component values before compression may have 
different color component values after compression. In this 
case, it is preferable to allow a margin of error. A margin 
of error may be allowed only when images are compressed by JPEG. 
The margin of error may be changed in accordance with the 
compression ratios of images. 
[0010] 

Various numbers of successive pixels of the same specific 
color can be adopted. On the other hand, it is rare for an image 
constituting a letter or the like to consist of two or three 
pixels in an image which consists of a plurality of pixels (for 
example, pixels in the form of a dot matrix) ; therefore, it serves 
the purpose if successive pixels of the same specific color 
equivalent in size to a letter or line can be extracted. As 
the whole size, or the number of pixels and resolution, of an 
image change, the number of pixels equivalent in size to a letter 
or line changes; therefore, the minimum number of successive 
pixels of the same color to be extracted may be changed in 
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accordance with the numbers of pixels and resolutions of images . 
Besides, not only successions in the horizontal or vertical 
direction but also successions in both the horizontal and 
vertical directions may be extracted. 
[0011] 

To extract image data characteristic of an artificial image, 
a plurality of successive pixels of the same color in the 
artificial image may be extracted. Most artificial letters, 
lines, patterns, etc. are in a single color. On the other hand, 
it is rare for a plurality of successive pixels of the same color 
to appear in a natural image on a display capable of expressing 
16, 700, 000 colors or so even if any pixels may appear to human 
eyes to be in the same color . Therefore, letters, lines, patterns, 
etc. can easily be extracted by extracting a plurality of 
successive pixels of the same color. 
[0012] 

If we define two pixels of the same color as two pixels 
with the same color component values, it is preferable because 
such pixels can be extr acted by simple processing . In some cases , 
however, it is preferable to regard two pixels as of the same 
color, allowing a margin of error of 1 or 2 of color component 
values. This is true of JPEG-compressed images mentioned 
earlier. Such a margin of error may be allowed for only 
JPEG-compressed images or changed in accordance with the 
compression ratios of images. 
[0013] 

To extract image data characteristic of an artificial image, 
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a scheme may be adopted that it is determined whether the tops 
or bottoms of the masses are aligned horizontally or not or whether 
the right or left edges of the masses are aligned vertically 
or not if there is a plurality of masses, each of which consists 
of pixels of a specific color, in a natural image. In this case, 
the above means for extracting artificial images finds a 
plurality of masses, each mass consisting of a plurality of pixels 
of the same specific color, and extracts the masses as artificial 
images if their edges are aligned horizontally or vertically. 
[0014] 

For example, the pixels at the top, bottom, right, and/or 
left edges of artificial letters are aligned horizontally and/ or 
vertically. On the other hand, a natural image may contain a 
photograph of a signboard containing letters and, even in this 
case, it is rare for the edges of the letters to align horizontally 
and/or vertically. Accordingly, an artificial image with edges 
without a slip of one pixel can be found in a natural image by 
finding masses, each of which consists of pixels of the same 
specific color, and checking to see whether or not their edges 
are aligned horizontally and/ or vertically . Thus, artificially 
made letters and the like can easily be extracted. Specific 
colors may be designated in advance or if there are two or more 
successive pixels of the same color, the color may be treated 
as a specific color. 
[0015] 

As described above, the present invention is to extract 
and exclude an artificial image or images, which are contained 
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in a natural image, from the natural image and process the natural 
image based on the natural image alone. The typical image 
processing of the present invention is that the tendencies of 
gradation of a natural image are grasped and revised by revising 
the gradients of each pixel. 
[0016] 

As the gradient of each of the color components of each 
pixel is prescribed, the tendency of gradation of each color 
component can be grasped. In addition, the components of 
brightness can be grasped from all the color components. Thus, 
the tendencies of gradation of various matters can be grasped. 
If the tendency of gradation of a matter is grasped, it can be 
determined whether the tendency of gradation is in accord with 
a predetermined desirable tendency of gradation or not. Thus, 
it can be determined whether the tendency of gradation should 
be revised or not. 
[0017] 

On the other hand, an artificial image is formed by pixels 
of specific colors alone or a mass of pixels of a single color. 
Thus, the tendencies of gradation of an artificial image differ 
from those of a natural image. Therefore, if the tendencies 
of gradation of a natural image containing an artificial image 
are grasped, the tendencies of gradation of the natural image 
cannot be revised appropriately. However, the present 
invention enables the extraction and exclusion of the artificial 
image or images from the natural image if an artificial image 
or images are contained in a natural image, the correct grasp 
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of the tendencies of gradation of the natural image, and the 
determination of whether to revise the tendencies of gradation 
of the natural image or not. 
[0018] 

When the gradients of pixels of a natural image are revised, 
the tendencies of color components of the natural image change, 
increasing or decreasing the tinge of, for example, red of the 
natural image, and the tendencies of components of brightness 
of the natural image change, increasing or decreasing the 
brightness of the natural image. Thus, if the tendencies of 
gradation of a natural image can be grasped, the tinge of red 
can be increased if it is weak and the brightness can be increased 
if the natural image is dark. Thus, the natural image can be 
revised so that its tendencies of gradation will be in accord 
with desirable tendencies of gradation. However, if the 
tendencies of gradation of a natural image containing an 
artificial image are grasped, it cannot be determined 
appropriately to what degree the tendencies of gradation of the 
natural image should be revised. 
[0019] 

However, the present invention enables the extraction and 
exclusion of the artificial image or images from the natural 
image if an artificial image or images are contained in a natural 
image, the appropriate grasp of how far the tendencies of 
gradation of the natural image are out of accord with desirable 
tendencies of gradation, and appropriate revision of the 
tendencies of gradation of the natural image. Various 
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tendencies of gradation including those of components of color 

and components of brightness can be adopted. 

[0020] 

A preferred example of the image processing is 
backlight-re vising processing . Backlight- revising 
processing is to grasp the tendency of brightness of a natural 
image and increase the brightness of the main part of the natural 
image if the main part (for example, the main subject in a 
photograph) of the natural image is dark. If the tendency of 
brightness of a natural image containing an artificial image 
is grasped, the tendency of brightness of the main part of the 
natural image cannot be grasped appropriately. Accordingly, 
it cannot be determined appropriately whether the natural image 
should undergo brightness-revising processing or not. It 
cannot be determined how far the brightness of the natural image 
should be revised, either. 
[0021] 

However, the present invention enables the exclusion of 
the artificial image or images from the natural image if an 
artificial image or images are contained in a natural image, 
appropriate determination of whether the natural image should 
undergo backlight-revising processing or not, and appropriate 
determination of the degree of the revision if the natural image 
needs backlight-revising processing. Thus, the present 
invention enables appropriate backlight-revising processing. 
Needless to say, the present invention may be applied to other 
kinds of processing than backlight-revising processing, because 
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it enables the extraction and exclusion of the artificial image 
or images from the natural image if an artificial image or images 
are contained in a natural image and the appropriate grasp of 
the tendencies of gradation of the natural image. One of said 
other kinds of processing is to widen the distribution of 
gradation of the natural image to change the contrast of the 
natural image if the distribution of gradation of a natural image 
concentrates on the center of the gradient scale. 
[0022] 

Before a natural image undergoes backlight-revising 
processing, it is usually determined whether the main subject 
in the natural image is backlighted or not . It can be determined 
by using various kinds of algorithm. If a natural image has 
a dark area which is surrounded by a bright zone, the dark area 
is usually regarded as backlighted. In most cases, however, 
an artificial image consists of pixels of the same brightness. 
Accordingly, if an artificial image is contained in a natural 
image, it cannot be determined correctly whether the natural 
image is of backlight or not . If an artificial image is contained 
in a natural image of backlight, the natural image may be regarded 
as of normal light. If an artificial image is contained in a 
natural image of normal light, the natural image maybe regarded 
as of backlight . If an artificial image is contained in a natural 
image, the degree of backlight-revising processing may not be 
grasped correctly. 
[0023] 

Accordingly, the present invention can be applied to a 
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scheme that it is determined beforehand by a 

backlight-image-checkup means whether a natural image taken by 
an image-data-acquisition means is backlighted or not. 
Specifically, with the above artificial-image-extracting means 
and image-processing means, image processing can be done to the 
natural image excluding the artificial image of images after 
a natural image containing an artificial image or images has 
been wrongly determined to be of backlight. 
[0024] 

Thus, even if the backlight-image-checkup means misjudges 
some natural images, it can be determined again by the following 
image processing whether they are of backlight or not. If 
backlight-revising processing is to be made to a natural image 
of normal light, doing backlight-revising processing of an 
appropriate degree to the natural image is equivalent to doing 
no backlight-revising processing to the natural image. Thus, 
natural images of normal light can be free of being affected 
by mis judgment by the backlight-image-checkup means. Namely, 
by combining the backlight-image-checkup means and the present 
invention, mis judgment by the backlight-image-checkup means can 
be made, up for. Of course, a natural image containing an 
artificial image or images may first undergo the 
artificial-image-extracting processing according to the 
present invention, then undergo the checkup of backlight, and 
further undergo the revision of backlight if it is of backlight. 
[0025] 

The present invention is to exclude the artificial image 
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or images from a natural image containing an artificial, or 
non-natural, image or images to process the natural image 
appropriately . Thus, the subjects of the image processing 
according to the present invention are natural images. 
Accordingly, a scheme may be adopted that a means for determining 
the kinds of images is first applied to an image and if it is 
determined to be natural, the above artificial-image-extracting 
means and image-processing means may be applied to the natural 
image. With this scheme, the present invention can be applied 
to only natural images which artificial images are excluded from 
and need to undergo image processing. Thus, unnecessary 
processing to images consisting of letters and charts, which 
do no need the application of the present invention, can be 
avoided. 
[0026] 

As described above, it serves the purpose if artificial 
images can be extracted in the present invention; accordingly, 
the artif icial-image-extractingmeans canbe embodied in various 
ways so long as it is capable of extracting artificial images. 
For example, it may be embodied so that it can calculate the 
quantity of a characteristic of the image from the image data 
of an image acquired by an image-data-acquiring means and 
determine whether the calculated quantity of the characteristic 
is close to a predetermined reference quantity of the 
characteristic or not . With this embodiment, it can be adjusted 
that the image should be determined to be artificial if how close 
the calculated quantity of a characteristic of an image is to 



13 



the reference quantity of the characteristic. Thus, the 
accuracy of determination can easily be adjusted. 
[0027] 

The quantities of various characteristics can be adopted. 
For example, if a part consisting of a plurality of successive 
pixels of one of the predetermined specific colors is to be 
extracted as described above, the quantity of such a 
characteristic can be adopted as the larger the number of pixels 
of other colors than the specific color in the successive pixels 
is, the larger the gap between the reference quantity of the 
characteristic and the calculated quantity of the characteristic 
is and the larger the gap between the gradients of pixels of 
other colors than the specific color and the gradients of the 
specific color is, the larger the gap between the reference 
quantity of the characteristic and the calculated quantity of 
the characteristic is. If a part consisting of a plurality of 
successive pixels of the same color is to be extracted, the 
quantity of such a characteristic can be chosen as the larger 
the number of pixels of other colors than a color accounting 
for the majority of the pixels is and the larger the gap between 
the gradients of pixels of other colors than the color accounting 
for the majority of the pixels and the gradient of the color 
accounting for the majority of the pixels is, the larger the 
gap between the reference quantity of the characteristic and 
the calculated quantity of the characteristic is. 
[0028] 

It is also possible to estimate by the quantity of a 
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characteristic if two or more parts, each consisting of a 
plurality of successive pixels of a specific color, are to be 
found and extracted as artificial images if the edges of the 
parts are aligned horizontally and/or vertically. For example, 
the quantity of a characteristic can be adopted which estranges 
itself from the reference quantity of the characteristic as the 
disalignment becomes larger if the edges of such two or more 
parts are aligned horizontally and/or vertically and there is 
such a part which is out of alignment. 
[0029] 

The technique of excluding the artificial image or images 
from a natural image containing an artificial image or images 
andprocessing the natural image as described above is not limited 
to the invention of a method, but effective as the invention 
of a device. Besides, some image-processing apparatus are used 
by themselves and others are built into other apparatus and used . 
Accordingly, various embodiments are included in the idea of 
the present invention. The idea of the present invention may 
be embodied as hardware or the same may be embodied as software. 
[0030] 

When the idea of the present invention is embodied as 
software for image-processing apparatus, the idea of the present 
invention exists on media wherein the software is stored and 
the idea of the present invention is used. Such media may be 
magnetic media, optical magnetic media, or any new media to be 
developed in future . Needless to say, the primary and secondary 
duplicated copies of the software fall into the scope of the 
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present invention . 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram of an image-processing apparatus 
in accordance with the present invention; 

Fig. 2 is a block diagram of an artificial- image-extracting 
module of Fig. 1; 

Fig. 3 is an illustration of the processing of the 
artificial-image-extracting module of Fig. 2; 

Fig. 4 is a flowchart of the processing of the printer 
driver (PRTDRV) of Fig. 1; 

Fig. 5 is an illustration of an example of processing to 
determine whether natural images are of backlight or not in 
accordance with the present invention; 

Fig. 6 is a flowchart of the processing to extract 
artificial images in accordance with the present invention; and 

Fig. 7 is a block diagram of another 
artificial-image-extracting module in accordance with the 
present invention . 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[0031] 

The embodiments of the present invention will be described 
in the following order. 

(1) Construction of the present invention 

(2) Construction of the artificial-image-extracting module 

(3) Printing 



16 



(3-1) Extraction of artificial image 
(4) Other embodiments 
[0032] 

(1) Construction of the present invention 
Fig. 1 is a block diagram of a computer 10 which is an 
image-processing apparatus in accordance with the present 
invention. The computer 10 has a CPU, a ROM, and a RAM and runs 
a prescribed program by using peripherals such as an HDD 15. 
Input devices for operation such as a keyboard 31 and a mouse 
32 are connected to the computer 10 through I/O ports for serial 
communication 19a . Adisplay 18 is also connected to the computer 
10 through a video board (not shown) . A printer 40 is also 
connected to the computer 10 through an I/O port for parallel 
communication 19b. 
[0033] 

The construction of the computer 10 shown in Fig. 1 is 
simplified and a personal computer of ordinary construction can 
be adopted as the computer 10. Of course, the computer which 
the present invention is applied to is not limited to a personal 
computer . The computer 10 is a desktop, but it may be a notebook 
computer or a mobile computer. The interface between the 
computer 10 and the printer 40 is not limited to the I/O port 
for parallel communication 19b, but may be a serial interface, 
SCSI, USB interface, wireless interface, or new interface to 
be developed in future. 
[0034] 

Although the computer 10 is the image-processing apparatus 
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in this embodiment, the printer 4 0 may be provided with an 
environment for running an image-processing program, acquire 
image data from a digital camera directly connected to it, and 
process the images represented by the image data. Of course, 
such a digital camera may process such images in the similar 
construction, or the image processing of the present invention 
may be carried out by distributed processing. Thus, various 
types of construction can be adopted. Moreover, the image 
processing of the present invention may be carried out by a 
compound device with the functions of a scanner and a printer. 
[0035] 

As shown in Fig. 1, a printer driver (PRTDRV) 21, drivers 
for input devices (DRV) 22, and a display driver (DRV) 23 are 
built in the OS 20 of the computer 10. The display driver (DRV) 
23 controls the display of images, etc. on the display 18. The 
drivers for input devices (DRV) 22 receive signals from the 
keyboard 31 and the mouse 32 through the I/O ports for serial 
communication 19a. 
[0036] 

The reference numeral 25 is an application program (APL) 
for retouch of color images, etc . While the application program 
(APL) 25 is running, the user uses the keyboard 31 and the mouse 
32 to print images out of the printer 40. In accordance with 
the user's instructions, the application program (APL) 25 reads 
RGB image data 15a from the HDD 15 into the RAM 14 and causes 
the display 18 through the display driver (DRV) 23 to display 
an image based on the RGB image data 15a. When the user operates 
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the keyboard 31 and the mouse 32, the contents of his operation 
are acquired through the drivers for input devices (DRV) 22 and 
interpreted and the application program (APL) 25 makes various 
kinds of processing such as printing and retouching in accordance 
with the contents of his operation. 
[0037] 

The RGB image data 15a are data in the form of a dot matrix 
expressing RGB color components in 256 gradients and capable 
of expressing about 16,700,000 colors. Of course, the present 
invention can be applied to various kinds of data such as data 
according to the JPEG standards, data according to Exif 2.2 
standards (Exif is a registered trademark of Japan Electronics 
and Information Technology Industries Association) , and data 
compatible with Print Image Matching (PIM is a registered 
trademark of Seiko Epson Corporation) , too. 
[0038] 

The application program (APL) 25 issues instructions for 
printing. Then, the printer driver (PRTDRV) 21 is driven, sends 
data to the display driver (DRV) 23, and displays a UI (not shown) 
to have the user input information necessary for printing such 
as page numbers. Once the user inputs the information through 
the UI with the keyboard 31 and the mouse 32, modules of the 
printer driver (PRTDRV) 21 are started and process the RGB image 
data 15a to generate data for printing. The data for printing 
are outputted to the printer 40 through the I/O port for parallel 
communication 19b and the printer 40 prints an image based on 
the data for printing. 
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[0039] 

More specifically, the printer driver (PRTDRV) 21 has an 
image-data-acquiring module 21a, a backlight-image-checkup 
module 21b, an artificial-image-extracting module 21c, an 
image-processing module 21d, and a printing-data-generating 
module 21e to print images. The image-data-acquiring module 
21a acquires pixel data from the RGB image data 15a and sends 
the pixel data to the backlight-image-checkup module 21b and 
the artificial-image-extracting module 21c. The 
backlight-image-checkup module 21b determines based on the 
brightness of the natural image represented by the RGB image 
data 15a whether the natural image is of backlight or not. 
[0040] 

In this embodiment, it can be determined whether the 
natural image represented by the RGB image data 15a sent from 
the image-data-acquiring module 21a is of backlight or not. It 
is also possible to determine whether the natural image 
represented by the RGB image data 15a, from which the artificial 
image or images are excluded by referring to data 15c on the 
position or positions of an artificial image or images in the 
natural image, is of backlight or not. The 

artificial-image-extracting module 21c refers to the RGB image 
data 15a sent from the image-data-acquiring module 21a and 
extracts the artificial image or images from the natural image. 
Then, the artificial-image-extracting module 21c prepares data 
15c on the position or positions of the artificial image or images 
and stores the data 15c in the HDD 15. 
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[0041] 

The image-processing module 21d revises the gradients of 
RGB of pixels in the form of a dot matrix and makes other kinds 
of image processing. The image-processing module 21d has a 
lightness-revising unit 21dl . The lightness-revising unit 21dl 
corrects backlight by referring to the data 15c on the position 
or positions of the artificial image or images, excluding the 
artificial image or images extracted by the 

artificial-image-extracting module 21c from the natural image 
representedby the RGB image data 15a, and revising the lightness 
of the natural image. Although details are omitted in this 
specification, other kinds of image processing may be made before 
and after the correction of backlight. In this case, it will 
do if the image-processing module 21d is provided with the 
functions to do such other kinds of image processing. 
[0042] 

The printing-data-generating module 21e transforms the 
image data after the processing by the image-processing module 
21d into data which can be printed by the printer 40. The 
printing-data-generating module 21e refers to a LUT 15d to 
convert the RGB data of pixels into CMYK data or the like 
(color-convertingprocessing) , converts the gradients of pixels 
into recording densities of ink drops (half-tone processing) , 
arranges pixel data in the order of use (rasterizing) , and does 
other kinds of processing. In this embodiment, the printer 
driver (PRTDRV) 21 has the artificial-image-extracting module 
21c and image-processing module 21d, and the 
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artificial-image-extracting module 21c extracts and excludes 
the artificial image or images from an natural image containing 
an artificial image or images and then the natural image is 
processed. The artificial- image-extracting module 21c will be 
described in detail below. 
[0043] 

(2) Construction of the artificial-image-extracting 

module 

Fig. 2 is a block diagram of the 
artificial-image-extracting module 21c. Fig. 3 is an 
illustration of the processing by the 

artificial-image-extracting module 21c. An example "A" of 
images formed by RGB data is shown in the left side in Fig. 3. 
There is a person in the center of the image "A" and logos in 
white letters are disposed in the four corners indicatedby broken 
lines. The artificial-image-extracting module 21c has a 
coordinates-comparing unit 24a and color-determining unit 24b. 
The coordinates-comparing unit 24a receives the RGB image data 
15a of a natural image which was determined to be of backlight. 
[0044] 

The coordinates-comparing unit 24a extracts an area which 
appears uniform to human eyes and extracts horizontal and 
vertical parts of the edge of the area. For this purpose, the 
coordinates-comparing unit 24a has an edge extractor 24al . The 
edge extractor 24al does edge-extracting processing to the RGB 
image data 15a. Edge-extracting processing means to calculate 
brightness of pixels from the RGB image data 15a and extract 
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an area where the change of brightness is great. With this 
processing, artificial images such as letters, lines, patterns, 
etc. can be extracted from natural images. 
[0045] 

The edge extractor 24al is a module to apply a so-called 
edge-extracting filter to the gradients of pixels. With the 
application of this filter, image data 15b of edges of areas, 
where the change of brightness is great, are prepared, pixels 
on the edges treated as "1", the other pixels treated as "0" . 
The edge extractor 24al stores the edge image data 15b in the 
HDD 15 . The enlarged view of a part (a bottom portion of a logo) 
of the edge image data 15b is shown in the upper right area in 
Fig. 3 . Edge pixels of "1" are shown in white and non-edge pixels 
of M 0" are indicated by hatching. 
[0046] 

The coordinates-comparing unit 24a refers to the edge image 
data 15b and compares the ordinates or abscissas of the edge 
pixels . The probability for the ordinates of top or bottom edges 
of areas to coincide with one another is almost nil unless the 
areas are artificial images . The probability for the abscissas 
of right or left edges of areas to coincide with one another 
is almost nil unless the areas are artificial images. The 
probability for the ordinates of successive top or bottom edge 
pixels of an area to coincide with one another is almost nil 
unless the area is an artificial image. The probability for 
the abscissas of successive right or left edge pixels of an area 
to coincide with one another is almost nil unless the area is 
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an artificial image. 
[0047] 

Thus, the coordinates-comparing unit 24a regards the areas 
as prospective artificial images if the ordinates of top or bottom 
edges of areas coincide with one another or the abscissas of 
right or left edges of areas coincide with one another. The 
coordinates-comparing unit 24a regards the area as a prospective 
artificial image if the ordinates of successive top or bottom 
edge pixels of an area coincide with one another or the abscissas 
of successive right or left edge pixels of an area coincide with 
one another. 
[0048] 

In the enlarged view in the upper right area in Fig. 3, 
there are the bottom edges of three areas at the ordinate "yl", 
whereas there is no edge pixels at the ordinate "y2" . Each of 
the three areas has successive bottom edge pixels at the ordinate 
"yl", whereas there is no edge pixel at the ordinate "y2" . 
Therefore, these areas are regarded as prospective artificial 
images . With this processing, a plurality of areas as indicated 
by broken lines in the left side in Fig. 3 are regarded as 
prospective artificial images. 
[0049] 

In this embodiment, the above prospective artificial 
images undergo the processing by the color-determining unit 24b, 
which is a module to determine whether the pixels of each of 
the prospective artificial images are in one and the same color 
or not. The color-determining unit 24b has a successive-pixel 
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extractor 24bl. When the color-determining unit 24b receives 
information about the positions of the prospective artificial 
images from the coordinates-comparing unit 24a, the 
successive-pixel extractor 24bl refers to the edge image data 
15b and extracts the prospective artificial areas. 
[0050] 

The enlarged view in the lower right area in Fig, 3 shows 
the areas extracted in this way. The pixels in each of these 
areas appear uniform to human eyes. If there exist a letter 
in one of these areas, pixels forming the letter should be 
contained in the area. On the other hand, if the prospective 
artificial images are not artificial, the probability for a 
plurality of successive pixels in each of the prospective 
artificial images to be in one and the same color is almost nil. 
Therefore, the color-determining unit 24b extracts the RGB data 
of the pixels of the prospective artificial images from the RGB 
image data 15a, compares the gradients of the pixels, and 
determines whether the pixels in each prospective artificial 
image are in one and 'the same color or not. 
[0051] 

If the pixels in each prospective artificial image are 
determined to be in one and the same color, said image is determined 
to be artificial . If the pixels in each prospective artificial 
image are determined to be in different colors, said image is 
determined to be natural . Data 15c on the positions of artificial 
images are stored in the HDD 15. By preparing the data 15c on 
the positions of artificial images, the artificial images are 
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extracted from the RGB image data 15a. 
[0052] 

In this embodiment, the above artificial images are 
excluded from the natural image and it is determined whether 
the natural image is of backlight or not. If the natural image 
is of backlight, the image-processing module 21d makes the 
revision of lightness to improve a backlighted object or objects 
in the natural image. Specifically, the 

backlight-image-checkup module 21b refers to the data 15c on 
the positions of artificial images to grasp the positions of 
artificial images, excludes the RGB data of artificial images 
from the RGB image data 15a, and determines whether the natural 
image is of backlight or not. If the natural image is determined 
to be of backlight, the lightness-revising unit 21dl revises 
the lightness of the natural image. 
[0053] 

The lightness-revising unit 21dl raises the brightness 
of a dark area or areas in the natural image so that the dark 
area or areas appear to be in normal light. Various well-known 
kinds of algorithm can be used for the revision of lightness. 
For example, y revision or the like can be adopted. It is also 
possible to calculate the parameter y by using the mean value 
of brightness of an ideal image of normal light and the mean 
value of brightness of the RGB image data 15a of the natural 
image and revise the lightness of the natural image toward the 
distribution of brightness of the ideal image of normal light. 
[0054] 
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(3) Printing 

When the natural image represented by the RGB image data 
15a is printed with the application program (APL) 25, the printer 
driver (PRTDRV) 21 makes the processing shown in Fig. 4. When 
the application program (APL) 25 issues instructions to print 
the natural image represented by the RGB image data 15a and 
parameters are inputted through a UI (not shown) , the 
image-data-acquiring module 21a of the printer driver (PRTDRV) 
21 acquires the RGB data of pixels of the natural image from 
the RGB image data 15a in Step S100. 
[0055] 

Then, the backlight-image-checkup module 21b determines 
whether the natural image is of backlight or not in Step S105. 
For this processing, various kinds of algorithm can be adopted. 
An example of this processing is shown in Fig. 5. A natural 
image of backlight usually has the main subject in its center 
and a lighter zone around the main subject as shown in Fig. 5. 
[0056] 

Because the RGB image data 15a prescribes the RGB 
components of each pixel, the components of brightness of each 
pixel can be calculated from the RGB components of said pixel 
( for example, brightness Y = 0 . 30R + 0 . 59G + 0 . 11B) and a histogram 
can be prepared as shown in Fig. 5. It is usual that the main 
subject in a natural image of backlight is dark and a zone around 
the main subject is bright; accordingly, the frequency of 
appearance of high brightness is high and the frequency of 
appearance of low brightness is relatively high in the histogram 
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of the natural image of backlight. Accordingly, a typical 
histogram of a typical natural image of backlight is prepared 
in advance and the histogram of the natural image represented 
by the RGB image data 15a are compared. If the two histograms 
are similar to each other, the natural image represented by the 
RGB image data 15a is regarded as a prospective image of backlight . 
[0057] 

The peripheral zone and the central area of the prospective 
image of backlight are divided into sections SI to S4 and sections 
S5 to Sll, respectively, and the mean value of brightness of 
each section is calculated. If the mean values of brightness 
of the central area are lower than the mean values of brightness 
of the peripheral zone, the prospective image of backlight is 
determined to be of backlight . As described above, various kinds 
of algorithm can be adopted for the identification of images 
of backlight . On the other hand, if a natural normal-light image 
with bright letters in its peripheral zone is checked by the 
algorithmof Fig . 5, the image may be determined to be of backlight . 
[0058] 

As described above, if a natural image containing an 
artificial image or images undergoes various examinations 
without excluding the artificial image or images from the natural 
image, the examinations may bring about wrong results. 
Accordingly, wrong judgments due to artificial images are 
prevented in the present invention. In this embodiment, there 
are Step S115 (extraction of artificial images) and Step 120 
(checkup for images of backlight) in addition to Step S105 
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(checkup for images of backlight) to prevent wrong judgments 
about backlight as shown in Fig. 5. 
[0059] 

More specifically, if the natural image represented by 
the RGB image data 15a is regarded as of backlight in Step S105, 
the artificial images contained in the natural image are 
extracted and excluded from the natural image in Step S115. 
During the extraction of artificial images, data 15c on the 
positions of artificial images are prepared as described in 
detail later. If the natural image is regarded as of normal 
light, the processing jumps from Step S115 to Step S135 as shown 
in Fig. 5. 
[0060] 

The backlight-image-checkup module 21b excludes the RGB 
data of artificial images from the RGB image data 15a based on 
the data 15c on the positions of artificial images and determines 
whether the natural image is of backlight or not in Step S120. 
If thenatural image is determined tobeof backlight, it undergoes 
backlight- revising processing inStepS130. If the natural image 
is determined to be of normal light, the processing jumps to 
Step S135. 
[0061] 

The lightness-revising unit 21dl makes the 
backlight-revising processing in Step S130. With this 
processing, the dark main subject in the natural image is given 
appropriate lightness and the natural image appears to be of 
normal light. In Step S135, other kinds of image processing 



29 



aremade. For example, color-balance adjustment , white-balance 
adjustment, and revision of contrast are made . These other kinds 
of image processing may be made before Step S105. 
[0062] 

Then, the printing-data-generating module 21e is started 
to make color-converting processing in Step S140. The LUT 15d 
stored in the HDD 15 is a table to match RGB data with CMYK data. 
Thus, the RGB data after the above image processing are converted 
into CMYK data for printing. In Step S145, the color of each 
pixel by CMYK gradients is converted into recording densities 
of CMYK ink drops and head-driving data are generated to attach 
ink to a printing medium at the recording densities. 
[0063] 

In Step S150, rasterizing is made to arrange the 
head-driving data in the order of use by the printer 40. In Step 
S155, certain information such as resolution is added to the 
data after the rasterizing to produce data for printing, which 
is outputted through the I/O port for parallel communication 
19b to the printer 40. Then, in Step S160, it is determined 
whether all the rasters constituting the image after the above 
image processing underwent the processes from Step S140 to S155 . 
The processes in and after step S140 are repeated until it is 
determined that all the rasters have undergone the processing. 
As a result, the printer 40 prints the image after the above 
image processing according to the above printing data. Namely, 
even if an original image is of backlight, the printer 40 prints 
the image which is corrected so that effects of the artificial 
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images are eliminated and the brightness of the main part of 

the image is increased. 

[0064] 

(3-1) Extraction of artificial image 

Now, a series" of processing of the artificial image 
extraction in Step S115 will be described according to the 
flowchart of Fig. 6. When extraction of an artificial image 
is started, the coordinates-comparing unit 24a receives the 
above-described RGB image data 15a and, in step S200, the edge 
extractor 24al applies the edge-extracting filter to the RGB 
image data 15a. With the application of this filter, image data 
15b of edges of areas are prepared, pixels on the edges treated 
as "1", the other pixels treated as "0". 
[0065] 

The coordinates-comparingunit 24a refers to the edge image 
data 15b in step S205 and scans edge pixels of "1" on the same 
ordinates or abscissas. Then, a border of an area surrounded 
by the edges is found by detecting a part in which there exist 
a plurality of pixels detected as edges .successively in one 
direction of coordinates and there exists none successively in 
the other direction. For example, in the view in the upper right 
area in Fig. 3, there is a plurality of edge pixels detected 
at the ordinate "yl" , whereas there is no edge pixel detected 
at the ordinate "y2". 
[0066] 

Further, there are successive edge pixels in the direction 
in which the value of the ordinate "y" reduces (in the direction 
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of an upper part of Fig. 3) . However, there exists none 
successively in the direction in which the value of the ordinate 
"y" increases (in the direction of a lower part of Fig, 3) . When 
these circumstances are grasped, it is found that there is a 
mass of edge pixels above the ordinate "yl" and there exists 
an area surrounded by the edge pixels . Thus, the area surrounded 
by the edge pixels is detected by giving the above processing 
vertically and horizontally to the pixels scanned for grasping 
the direction in which the edges exist successively. There are, 
of course, various methods for scanning edge pixels . For example, 
a line of rasters of the RGB image data 15a may be scanned as 
a whole. Alternatively, a certain number of pixels as a unit 

(for example, an area indicated by broken lines in the left part 
of Fig. 3) may be scanned. 

[0067] 

Thus, when the area surrounded by the edge pixels is 
detected, it is found that the edge pixels existing at the end 
portion thereof form the border of the area. Accordingly, in 
Step S210, the coordinates-comparing unit 24a compares the 
ordinates or a abscissas of the edge pixels to see if they are 
coincident and determines if an area surrounded by them can be 
prospective artificial images. To be more specific, the 
coordinates-comparing unit 24a regards the area as a prospective 
artificial image if the ordinates of top or bottom edges of areas 
coincide with one another or the abscissas of right or left edges 
of areas coincide with one another. Also, the 
coordinates-comparing unit 24a regards the area as a prospective 
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artificial image if there are numbers of pixels in which the 
ordinates of successive top or bottom edge pixels of areas 
coincide with one another or the abscissas of successive right 
or left edges of areas coincide with one another. Further, in 
the former case, an artificial image such as letters in which 
a plurality of areas exist successively at certain pixel 
intervals is assumed. Also, in the latter case, an artificial 
image such as lines in which a plurality of areas do not exist 
successively but numbers of pixels exist successively in one 
area is assumed. 
[0068] 

In Step S210, when prospective artificial images are 
extracted by determining whether or not the ordinates or 
abscissas of edges are coincident, in Step S215, information 
on positions of the prospective artificial images is prepared 
and is given to the color-determining unit 24b. When no 
prospective artificial image is extracted in Step S210, the 
processing jumps to Step 220. In Step S220, it is determined 
whether or not all the acquired RGB image data 15a underwent 
the above extraction of prospective artificial images* The 
processes in and after Step S205 are repeated until it is 
determined that all the data have undergone the above extraction 
of prospective artificial images. 
[0069] 

When prospective artificial images are extracted as 
described above, in and after Step S230, the color-determining 
unit 24b determines whether the pixels of each of the prospective 
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artificial images are in one and the same color or not to extract 
the artificial images. For this purpose, the successive-pixel 
extractor 24bl of the color-determining unit 24b extracts 
successive edge pixels in step S230. Namely, a border portion 
of the above prospective artificial image is extracted according 
to successive edges, and pixels on and inside the border portion 
are grasped. Since the border portion and the inside thereof 
form one area in the image, in Step S235, RGB data of pixels 
corresponding to those pixels are extracted from the above RGB 
image data 15a and it is determined whether those pixels are 
in one and the same color or not. 
[0070] 

If the pixels are determined to be in one and the same 
color in Step S235, the data about the positions of those pixels 
are stored in the HDD 15 as data 15c on positions of artificial 
images. If the pixels are determined to be in different colors 
in Step S235, the processing jumps to Step S245. In Step S245, 
it is determined whether or not all the prospective artificial 
images underwent the above color determining processing. The 
processes in and after Step S230 are repeated until it is 
determined that all the prospective artificial images have 
undergone the color determining processing. According to the 
above processing, the artificial images are extracted from the 
RGB image data 15a and the data 15c on the positions of the 
artificial images are prepared. 
[0071] 

(4) Other embodiments 
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In the first embodiment, artificial images are extracted 
on the condition that ordinates or abscissas of edges are 
coincident with one another and that the common area is in one 
and the same color . In the present invention, however, it serves 
the purpose if artificial images are extracted from natural 
images containing the artificial images and image processing 
is performed by excluding such artificial images. Therefore, 
various other constructions can also be adopted. Fig. 7 is a 
block diagram of the artificial-image-extracting module 210c 
according to the second embodiment . 
[0072] 

In the second embodiment, a part of the construction of 
the artificial-image-extracting module 210c is different from 
the one in the above first embodiment. Other constructions, 
namely, the module construction of PRTDRV 21 in Fig. 1 and the 
construction of the coordinates-comparing unit 24a are the same 
as those in the first embodiment. The above 
artificial-image-extracting module 210c has a specific 
color-determining unit 240b in stead of the color-determining 
unit 24 0b . The specific color-determining unit 24 0b determines 
whether pixels in each of areas appearing uniform to human eyes 
are in a prescribed specific color or not. 
[0073] 

For this purpose, the specific color-determining unit 240b 
has a successive-pixel extractor 240bl, which refers to the edge 
image data 15b and extracts portions therein comprising 
successive pixels regarded as "1" and an area surrounded by the 
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successive portions. Namely, when the coordinates-comparing 
unit 24a prepares the edge image data 15b by the same processing 
as in the first embodiments, extracts prospective artificial 
images and gives information on the positions of the prospective 
artificial images to the specific color-determining unit 240b, 
the successive-pixel extractor 24bl extracts the uniform 
prospective artificial areas. 
[0074] 

The specific color-determining unit 240b determines 
whether the color of the area extracted by the successive-pixel 
extractor 240bl is one of the specific colors or not. When the 
color of the extracted area is one of the specific colors, the 
area is regarded as an artificial image and data representing 
its position is stored in the HDD 15 as the data 150c on the 
position of the artificial image. The specific color described 
above is specified by specific color data 150e stored in the 
HDD 15 in advance, and the specific color data 150e are, in other 
words, gradients of the pixels of RGB color components 
representing specific colors. 
[0075] 

Namely, in the present embodiment, colors frequently used 
for artificial images are grasped as specific colors in advance . 
When the pixels of the area surroundedby edges are of this specific 
color only, the area is regarded as an artificial image. If 
it is a natural image, even a portion which appears to have one 
and the same color to human eyes actually has different colors 
at pixel level (the gradients of the pixels are different) . 
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Considering the above, it is almost impossible for a certain 
area to be of a specific color only. Therefore, if pixels of 
an area extracted by the successive-pixel extractor 240bl is 
of a specific color only, this area can be extracted as an 
artificial image. 
[0076] 

Artificial images such as letters and lines are generally 
prepared by various application software, and most of such 
application software have several to scores of default colors 
to be chosen for those artificial images. Therefore, as to an 
example of specific colors, a construction using those default 
colors and the like can be adopted. To be more specific, in 
RGB color components whose gradients range from 0 to 255, the 
colors (R, G, B) = (0, 0, 0), (0, 0, 64),... (255, 255, 255) 
expressed by either of the color component values of 0, 64, 128, 
192, and 255 can be adopted. The specific color data 150e shown 
in Fig. 7 is an example of the data of such colors. 
[0077] 

Of course, it is possible to adopt other values as specific 
colors. For example, the values may be varied as 0, 32, 64, 
96, 128, 160, 192, 224, 255, and the colors expressed by the 
combination thereof may be adopted. Alternatively, a color 
whose color component values are the same (achromatic color) 
may be used as a specific color. Of course, a user may choose 
any color as a specific color, and various other constructions 
can also be adopted. 
[0078] 
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When the specific color-determining unit 240b prepares 
data 150c on the positions of artificial images, the 
backlight-image-checkup module 21b extracts artificial images, 
excludes the RGB data of the artificial images from the RGB image 
data 15a based on the data 150c on the positions of artificial 
images and determines whether the image is of backlight or not 
again. If the image is determined to be of backlight, it 
undergoes backlight-revising processing by the 
lightness-revising unit 21dl. Therefore, even in the case of 
an image containing artificial images, mis judgment by the 
backlight-image-checkup means is prevented, and the degree of 
backlight revising can be grasped correctly. 
[0079] 

In the embodiments described above, the judgement by the 
coordinates-comparing unit 24a and the judgement by the 
color-determining unit 24b or the specific color-determining 
unit 240b are combined. Of course, however, these three types 
of judgement may be all combined or only one judgement may be 
satisfactory, and any combination thereof can be adopted. 
Further, in a construction adopting the judgement by the 
color-determining unit 24b or specific color-determining unit 
240b only, it is necessary to extract a uniform area by the edge 
extractor 24al, etc. 
[0080] 

Since it serves the purpose if the edge extractor can 
extract an area which appears uniform to human eyes, edges may 
be estimated not according to the brightness as described above 



38 



but according to gradients of pixels of each color component. 
Of course, various constructions can be adopted so long as they 
can extract an area which appears uniform to human eyes. 
[0081] 

Further, in the above embodiments, it is required that 
the ordinates or abscissas of the edges of areas exactly coincide 
with one another and the pixels of areas are in one and the same 
color. However, some allowance may be made for the judgement 
without demanding such strict conditions. For example, a 
process can be adopted in which functions are defined such that 
the more the ordinates or abscissas deviate the greater the value 
becomes or the more the difference in colors of the area grows 
the greater the value becomes, and the image is determined as 
an artificial image when the function value is below a certain 
threshold value. Of course, it may be so constructed that a 
prescribed threshold value is set and, when difference between 
the coordinates or gradients of pixels is below the threshold 
value, they are regarded as coincident. Thus, when allowance 
is made for the judgement, precision in extracting artificial 
images can be controlled. Further, it becomes possible for the 
present invention to be applied to the format like the one used 
for an image by JPEG in which two pixels with the same color 
component values before compression may have more different 
colors after compression. 
[0082] 

Further, in the present embodiment, it serves the purpose 
so long as an artificial image is extracted during the image 
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processing, and the artificial image may be extracted in various 
other ways . For example, data on a position where an artificial 
image is imbedded may be written in a header portion of data 
according to Exif 2.2 standards or data compatible with Print 
Image Matching so that the presence of the artificial image and 
its position are grasped by referring to such data. According 
to such a construction, the artificial image can be extracted 
by very simple and high-speed processing. 
[0083] 

Further, if an artificial image can be extracted from a 
natural image, it makes it possible to exclude the artificial 
image to perform appropriate image processing. Accordingly, 
the present invention can be applied to various image processing 
in addition to the backlight- revising processing described above . 
For example, if the present invention is applied to revision 
of contrast in which brightness distribution is expanded or 
narrowed when the value area of brightness of the image is 
extremely narrow or expanded, the judgement as to whether or 
not the revision of contrast should be performed or the degree 
of revising the contrast can be appropriately made . In addition 
to this, when the present invention is applied to color fogging 
correction which corrects the gradients of color components when 
color fogging occurs in which gradients of the color components 
are distributed over an extremely greater value range or 
extremely smaller value range, the judgement as to whether or 
not the color fogging correction should be performed and the 
degree of correcting color f oggingcanbe appropriately achieved. 
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[0084] 

Further, in the present embodiment, it serves the purpose 
if image processing is not inappropriately affected by an 
artificial image when a natural image contains the artificial 
image . Therefore, it may be determined in advance whether image 
data are of the natural image or not. For this purpose, it may 
be determined, for example, whether the brightness histogram 
of the image has an expanded portion or not, and determined that 
it is a natural image when the histogram has a certain expanded 
portion. Alternatively, data indicating that an image is 
natural or not may be written in the header of image data to 
make judgement according to such data. Thus, various kinds of 
constructions can be adopted. According to such constructions, 
it is possible to avoid applying the present invention to images 
of texts, charts and the like to which the present invention 
doesn't have to be applied. 

Although the present invention has been described and 
illustrated in detail, it is clearly understood that the same 
is by way of illustration and example only and is not to be taken 
by way of limitation, the spirit andscope of the present invention 
being limited only by the terms of the appended claims. 
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